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Introduction 

The purpose of this report is to explore an extension of schema theory as 
a method of analyzing solutions to an ill-structured problem. The immediate 
perspective taken is from cognitive science, while the lung-term perspective is 
from applied measurement, specifically from work on methods of assessing complex 
behaviour. Issues in the administration of schools serve as a vehicle for 
exploration of methodological concerns. Although the context for the study is 
adults responding to a case study, the implications of the method extend to 
student responses to a variety of stimuli designed to assess some of the more 
complex (i.e., "higher order") outcomes of education. 
The objectives of this report are: 

1. To present a methodology for the analysis ana comparison of responses to an 

ill- structured social problem. 

2. To examine the potential of these methods to reveal differences in suDject 

responses related to experience and e. pertise. 

3. To discuss issues in the application of the methods. 

The results reported herein are an extension and elaboration of one aspect 
of Nagy, Allison, Allison & Moorhead (1990). Subjects varying in experience in 
the elementary school principalship were presented with a case study which 
related a typical problem that might face a principal in his/her working day. 
Responses to this task were analyzed using a method: 

(a) based initially on the work of Voss (Voss, Greene, Post & Penner, 1983), 

(b) refined through a series of exploratory studies (see Nagy. Allison, Allison 

& Moorhead, 1990), and 

(c) extended through application of rating scales based on the work of Biggs and 

Collis (1982). 
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Both schema theory and the concept of an ill-structured problem have 
evolved in recent years as the interests of researchers in memory and problem 
solving have turned to complex phenomena. To date, most work on ill-structured 
problems has been based on adaptations of and analogies to artificial 
intelligence, in the form of production rules and flowchart- like portrayals. 
Schema theory offers a potentially profitable alternative. The essence of the 
procedure is to build a collective story-line, across subjects, capturing the 
variety of responses to the problem, including values exhibited, perspectives 
taken, and actions planned. Within this collective framework, then, individual 
responses to the problem can be highlighted and compared. The motivation for 
this particular study stems from the inability of schema theory to capture 
sufficient detail in problem solutions without becoming unwieldy. 

Literature 

The literature forming the background to this study comes from several 
sources. Among these are work on memory for complex phenomena, the solution of 
ill-structured problems, expert -novice differences, problem solving in school 
administration, and assessment of complex learning outcomes. As more detailed 
discussion is available in e-rlier writings (Nagy and Allison. 1983; Allison and 
Nagy, 1989; Nagy and Moorhead. 1990) I present here only a brief overview. 
Memory for Comp l ?X Phenoaena 

Researchers have posited several varieties of complex entities as 
organizing p-inciples in attempting to understand memory for stories and 
sequences of events. Anderson, Spiro and Anderson (1978) describe schema theory, 
one version of such an organizing principle. Schemata act as mental structures; 
they incorporate general knowledge, and are more abstract than the particulars 
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of a given situation. Interpretation of memory data involves matching elements 
in a specific situation to generic slots or placeholders. They report a study in 
which subjects read a restaurant story and a grocery-store story involving 
purchase of the same foods. The expectation was that the subjects' internal 
restaurant schemata, the expectations of the sequence and content of courses 
served in a fine restaurant, would impose structure on memory and result in an 
increase in memory for the foods mentioned. The results confirmed their 
expectation, and support the hypothesis that complex material is stored by 
complex mechanisms. 

Cognitive theorists have debated the psychological status of schemata. 
Aioa and Hasher (1983) argue that the evidence shows that stored memories are 
richer than the selected subset predicted by schema theory. This suggests that 
it is appropriate to view schema theory as a method of imposing order on 
complexity, not necessarily involving any s'.rong assumptions concerning the 
nature of human memory. This perspective, adopted here, gives the method the 
status of a portrayal technique, an heuristic device useful for imposing order 
on data. 

As a device for imposing order and examining differences, schema theory 
holds promise. Tor example, Schallert (1982) notes that schemata evolve; that 
is, they become more elaborate and specific with experience. This suggests that 
an examination of the details of story- lines across individuals might be used to 
highlight differences in specificity or sophistication, differences which in turn 
might be linked to experience or expertise. 

There is a difficulty with such a perspective, however: how to decide what 
constitutes • more complete or more sophisticated version of a schema. For 
example, Horton and Mills (1984) reviewed the literature on human learning from 
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a schemata perspective, using a levels-of -processing framework, they concluded 
that such an approach is plagued by the lack of an independent definition of 
depth of processing. Thus, a present limitation to the technique is reliance on 
subjective decisions concerning the adequacy or quality of particular pieces of 
data. In this project, we have dealt with this difficulty in twc ways: first, we 
have used a panel of expert judges to rate quality of our subjects' responses; 
second, we have adapted Biggs and Collis' "structure of learning outcomes" scale 
to assess the complexity of respondents' productions. 
T1 l-Str»rfAired P roblems 

Frederiksen (1984) summarized Simon's original distinction between well- 
structured and ill -structured problems. The characteristics of ill- structured 
problems include greater complexity, less definite criteria for deciding if a 
solution has been reached, lack of complete information, absence of a "legal move 
generator" , and nc convenient list of accepted procedures. They also have higher 
verbal content and are more context dependent. Most "real-life" problems would 
be classified as ill -structured. 

Methods for the analysis of ill-structured problems have evolved from the 
production rule or flowchart methods designed for well -structured ones (e.g., 
logic puzzles, chess) . Voss and Post (1988) noted that the method chosen for the 
analysis of ill-s.ructured problems reflects the theoretical concerns of the 
investigators. A variety of approaches has been developed to meet "arious 
concerns. For example, Larkin (1980) , working with physics and algebra problems, 
has found that large-scale units are useful in the analysis of problem solving 
in such domains. Voss, Greene, Post & Penner (1983), investigating how subjects 
would deal with the lack of productivity of the Soviet agricultural system, 
categorized statements as pertaining to goals or reasoning. Goal statements dealt 
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with relatively global moves such as identification of major issues and 
subproblems, while reasoning statements dealt with the analysis within the 
structure of these subproblems. Finally, Lawrence (1988), in the context of 
judicial decision making, developed a model using elaborate if... then 
statements, in an attempt to capture a priori perspectives ("frames of 
reference"), which correspond, according to Voss and Post (1988), to the 
magistrates' courtroom schemata. 
gv pgrt-Novice Dif ferences 

Glaser and Chi (1988) have provided a survey of the literature on expert- 
novice differences in a large variety of problem solving contexts. Within the 
ill-structured framework, Voss et al (1983) demonstrated general characteristics 
of expert solutions: experts did not articulate their highest level plans; they 
did not speak of a general solution strategy; they did not begin with a well- 
developed plan, but developed one in reviewing the problem; expert plans could 
be classified as either problem conversion (to one they could solve) or problem 
decomposition; experts tried for one general solution to solve the problem, using 
a lot of reasoning; novices tended to deal with low level subproblems and use 
little reasoning. In solution activity, only experts examined the implications 
of their solutions. Experts spent a lot of time on argumentation. 
Administration af p ™H PTn Solving 

Two lines of research have examined the school principalship from a problem 
solving perspective. From an administrative perspective, Leithwood and Stager 
(1986) reported differences in principals' sorting of problems, their solution 
strategies, and the influences on th.ir strategies and priorities. Their results 
showed that better principals use group solutions more often, and in different 
ways; they emphasise programs over buildings and relationships, and give fuller 
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rationales. Better principals tackle fundamental rather than superficial 
problems, are more reflective and seek clarification more, and list more specific- 
sources of information Better principals are aware of "problem solving" as a 
definite activity and they enjoy it and are confident at it. 

Leithwood and Stager (1987) analyzed expert and non-expert protocols on 
school problems, finding similarities in the two groups on clear problems and 
differences on unclear ones. In the latter, nonexperts had more interpretation 
statements, and fewer goals and solution processes. The experts thought of 
school and system consequences; nonexperts thought of personal consequences. 
Experts expressed belief in the power of rational thought; nonexperts didn't. 
Experts were more specific in statements and assumptions, and cited more relevant 
anecdotes. Experts planned moro; nonexperts paid little attention to planning. 

The present report is one in a series that has taken a cognitive rather 
than administrative perspective on principals' problem solving; more detail can 
be found in Nagy, Allison, Allison and Moorhead (1990). Brietly, and with the 
benefit of hindsight, the following points of Progress guide the present 
methodology; 

First, simulations differ from real-world scenarios. Cases are limited in 
length and complexity while, for example, in Voss' Soviet agriculture 
problem, respondents are limited only by their own knowledge. Different 
patterns of reasoning emerge, and different category systems are required. 
(Nagy and Allison, 1988). 
Second, ccntent-free counts of abstracted statement types (e.g., reasoning, 
action) do not alone capture differences in quality. In addition, 
content-based categories (e.g., individuals consulted, information sought, 
specific actions taken) are required (Nagy and Moorhead, 1990). 
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Third, in addition to the usual think-aloud training (Ericsson & Simon, 1984), 
substantial stage-setting and probing of subjects is required to deal with 
-things left unsaid" and individual differences in speaking style (Allison 
and Nagy, 1989). 

Fourth, simulations, and ill-structured problems in jmwral. vary in their 
degree of structure. Schema theory will be more useful for more complex 
problems such as the present, and of limited application to simple 
problems such as "what would you do if..." (Nagy, 1990). 
Fifth, as Alba & Hasher (1983) suggest, schema theory cannot capture all 
worthwhile detail of case study responses. If, in building of the schema, 
sub-category formation stops at a manageable level, the placeholders or 
categories do not demarcate all useful differences across responses (Nagy, 
Allison, Allison & Hoorhead, 1990). 
AscP.ssment implex Learning Outcomes 

Although this particular study does not deal with student response to 
learning, one of the goals of the larger research program is student assessment 
in complex areas. It is increasingly recognized that the more complex goals of 
education lack appropriate assessment methods (Archbald and Neumann, 198«). 
Despite the advantages of schema theory in dealing with complexity, there is a 
limitation. For example, the schema developed for the present study was 
structured into as many as five hierarchical levels, but still failed to capture 
a satisfying level of detail in the content of what was said. Statements 
revealing vastly different degrees of insight and complexity were categorized 
together, but attempts to subdivide categories further became unwieldy and were 
abandoned. 

One possible solution *o this difficulty lies in an Adaptation of the work 
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of Biggs and Collis (1982), who have identified five levels of response to tasks 
presented to students. Although they limit their work to problems with enough 
structure for there to be known answers, it appears relatively easily adaptable 
to the case under investigation. 

Each of these five levels is described with respect to memory capacity, 
mental operations used, consistency of reasoning and nature of closure brought 
to the question. In terms of structural complexity, they describe their levels, 
from lowest to highest, as follows. 

1. Prestructurai responses work from irrelevant or erroneous data to one 

response . 

2. Unistructural responses work from one relevant and given datum to one 

response . 

3. Multistructural responses work from several relevant and sjiven data to one 

response. 

4. Relational responses generalize within the given data to produce one 

response . 

5. Extended abstract responses generalize rithin both given and hypothetical 

data and produce or allow for more than one response. 
In the present study, the utility of a modified version of this scale was 
investigated. 

Me' nod 

Subjects 

Subjects were chosen from volunteers recruited from si:, moderate to large- 
sized county school boards in southern Ontario. The sampling plan called for 
eight subjects from each of four experience levels in the elementary school 



ERIC 



10 



10 



9 

ERIC 



principalship: Aspirant (0 years), Novice (1-2 years), Seasoned (10-15 years) 
and Veteran (20 plus years). (In the discussion and tables that follow, these are 
sometimes referred to as Levels 1 through 4.) Three transcripts were lost, but 
in two cases, this was discovered in time for the subject and interviewer to 
attempt a joint reconstruction of the case solution. Thus, there are data from 
31 subjects, 8, 8, 7, and 6 respectively, from the four experience levels. Half 
the inexperienced group were female; the remaining 27 were all male. 
Data Collection 

Subjects in groups of four spent a day with the research team. They 
participated in a group discussion, responded individually to a related 'act- 
finding activity and to the case study, filled out several paper-and-pencil 
instruments, and were interviewed extensively. Only the case study results are 
reported here. 

Each group began with a 30 minute warm-up and group discussion, and then 
individual research team members each took one subject. These individual sessions 
began with a fact finding activity in which subjects were told they would be 
asked to deal with a cas and were then allowed to ask whatever questions they 
wished about the school setting that formed part of the case. Next, subjects 
undertook training in the think aloud process using suggestions from Ericsson and 
Simon (1984). and then went on to the case study. First, subjects were asked to 
read the case iloud. interjecting their thoughts as they read. Then, they were 
asked to think aloud about how they would solve the process. Finally, they were 
asked to recall their thought processes . 

The degree of separation between reading and solving varied across 
subjects, as did the degree of thoroughness with which the final recall was 
treated. The process took between 50 and 100 minutes for each subject. 
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The Case 

The presented case describes a conflict between a school staff and school 
librarian. The situation, presented in the story to a principal new to the 
building, involves elements of supervision, curriculum, policy, interpersonal 
relations, physical plant, budget and supply, and staff attitudes. The librarian 
is Miss MacDonald. the principal is Pat Jones, and d» other named protagonist 
is Miss Green, a first-year teacher. Tne full case appears as Appendix A. 

Analysis 

The Schema 

The essence of the procedure used CO analyze the data is to build a 
collective story-line, across subjects, capturing the variety of responses to the 
problem, including values exhibited, perspectives taken, and actions planned. 
Within this collective framework, then, individual responses to the problem can 
be highlighted and compared. An eartit - version of this collective story-line 

or schema was initially reported in Nagy, Allison. Allison and Hoorhead (1990). 

Issues encountered with that analysis, leading to the extension reported here. 

can be briefly summarized: 

First, segmentation of the protocol into units for analysis, determination of 
when one thought unit ends and another begins, depends on striking a 
balance between capturing detail and producing a manageable category 
system. 

Second, organir-clon of the category system can facilitate its use. Seven major 
areas of problem content arose from the case, but ma.vy categorization 
decisions were difficult. Difficulties were minimized by organizing the 
categories from more global r-he problem solving process) to acre specific 
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areas (the librarian). It was a more precise and less taxing task to 
postpone categorization only when a statement clearly fit a smaller 
category than to postpone when the statement only fit a larger category. 

Third, as mentioned above, a hierarchical category system becomes unwieldy 
before all data elements in a single category are sufficiently homogeneous 
to distinguish levels of experience and expertise in the data set. 

The Ratings 

Assigning statements to levels of the schema alone did not capture 
sufficient detail; an additional analysis was developed to deal with this issue. 
This system, modelled on Biggs and Collis (1982), uses a 4-point scale to rate 
the amount of information dealt with by each subject within the statements 
assigned to each schema category: 

1: makes one or two points without supporting argument or elaboration 

2: makes one or two points with supporting argument or elaboration 

3: makes two points and relates them to each other with supporting 

argument or elaboration 
4: makes more than two points and relates them to each other with 

supporting argument or elaboration. 
The interpretation of these definitions varied with the nature of the 
schema category. In particular, it was not always clear whether the rating scale 
addressed quality or quantity. Certainly, there were times when the quality of 
what was said was not captured by the system. There were other ti^s when simple 
quantity (i.e., number of ideas) was misleading, particularly if the subject 
simply listed several unrelated points. In practice the counting of "points" 
from the above definitions sometimes took second place to the following 
retrospectively reconstructed implicit definitions: 
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1: just mentioned in passing 
2- making a routine point or observation 
3: synthesizing several ideas with evidence of thought 
4: marshalling a major argument or rationale, 
•jry Re porting 

The reporting in this paper covers six issues: first, the collective schema 
for the problem solution; second, the relationship between statement rating 
levels and experience; third, the relative emphasis within the schema as a 
function of experience; fourth, how the schema differentiates across experience; 

i_ Across expertise; and sixth, how the ratings 

fifth, how the schema differentiates across 

differentiate across expertise. 



Results 



The Schema 

Segmentation and categorization of the 31 transcript; . followed by 
hierarchical organization of the category system, produced the schematic 
representation displayed in Table 1. The narrative version of the schema, 
produced from the detailed classification system, conveys better than the tabular 
display an overall flavour of the collective approach of the respondents. 

. . , Kfl ctih<5tantive issues of the case are referred to 
Readers with an interest in the substantive 

Appendix B. 

Insert Table 1 about here 

The focus of much of the rest of this report is on the methodological 
issues raised by the analysis, and on the potential of such methods for 
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examination of complex problem solving. Discussion of these issues cannot take 
place in the abstract, however, and some detail concerning the administration of 

schools is included. 

The reliability of statement categorizations w;,s checked by having two 
independent raters categorize all the statements from a randomly -chosen two of 
the 31 subjects. Consistency was not as high as might be hoped for. All three 
raters agreed exactly in only 43% of the cases, but this figure rises to 56% if 
the criterion is agreement on the first two digits of the categorization. If we 
look for agreement between any two of the three raters, the figure for perfect 
agreement is 72%, and for agreement to two digits, 86%. This analysis was done 
with little training and no discussion, and thv represents a lower estimate of 
the reliability possible with this type of analysis. 
The Ratings 

One rating on the 4-point scale was assigned to each set of statements made 
by one respondent categorized as belonging to the same schema category. Table 2 
summarizes the numbers of each statement type as a function of experience. As can 
be seen, the uost substantial difference with experience is the number of "3" 
statements made. 

******* i. ***************** 

Insert Table 2 about here 

************************* 
T..e reliability of the ratings assigned to the statements was checked by 
having two independent raters rate a randomly chosen 100 of the 809 rated 
statements. Any pair ~Z raters tended to agree about 60% of the rime, but at 
least two of the three agreed in 96% of the cases. Again, this analysis was done 
with neither training nor discussion, and the potential exists for considerably 
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better reliability. 
The Role of £xpgXl£Qg£ 

Table 3 was produced by totalling all the rating points assigned to 
respondents in each experience level for each of the seven broad divisions of the 
schema, and then converting to percentages. The results have been dubbed 
-relative emphasis" , in deference to the fact that whether the ratings on the 4- 
point scale measure quality or quantity is a debatable point. Some slight trends 
can be seen across experience levels: more experienced subjects talked slightly 
more about the problem solving process; they spent more time on general personnel 
issues, and less on the librarian specifically. Of note is the fact that the 
three categories oi. the community the school system, and the school received 
leis than 10% of the emphasis. 

Insert Table 3 about here 

Further trends related to experience can b* found by examining emphasis in 
the detailed schema as a function of experience. Table 4 gives a breakdown of 
total rating points by experience for those categories that showed interesting 
differences across experience levels, as Judged subjectively. Under Problem 
Solving, the first pair of entries, 1.2.1.1 and 1.2.1.2, shows that less 
experienced subjects preferred to think problems through alone, while those with 
more experience preferred to brainstorm. The next pair of entries demonstrates 
that the Veterans related many incidents from their personal pasts, but did not 
reflect on the value of experience as much as the Seasoned group . The third pair 
of entries display that inexperienced respondents were quick to recognize that 
information was needed, but that they did not have the strategies available from 
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experience. Finally, the last pair of entries shows that less experienced (i.e. , 
more recently trained) subjects elected more to develop action plans for 
themselves and ownership of problems for their staffs. 

************************* 
Insert Table 4 about here 
************************* 

The entry under category 4, The School, is included to show how little 
attention was paid by must respondents to the existence of a vice-principal in 
the school. Under the Library, Table 4 includes figures showing that less 
experienced subjects tended more to refer to the provincial library policy, that 
they were more willing to diagnose the problem, and that they tended to see 
professional development for the entire staff as part of the solution to the 
problem. The lack of attention to staff professional development by the two more 
experienced groups is, in fact, remarkable. 

UnJer category 6, Personnel, the more experienced people spoke much more 
about how a new principal ought to establish him/herself in a school, about 
planning for meetings, and about team building. Finally, while most respordents 
said something about finding out how Miss MacDonald feels about things, those 
with more experience emphasized this more. On the other hand, those with less 
experience spoke more of her professional development. 
The Role of Expertise 

As part of the project, three professors of educational administration were 
given summaries of the transcripts and asked to rate each performance on a 10- 
point scale. The summaries were in the respondents' own words, but gave only the 
actions taken and immediately surrounding text. Only summaries were provided due 
to the length of the transcripts; with the actual reading of the case study text 
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removed, the responses run to soae 200 single -spaced pages. Based on the mean 

ratings, the five lowest scoring and five highest scoring subjects were selected. 

The lowest scoring were three Novices and two Veterans, while the highest scoring 

were two Novices and three Seasoned principals. 

Table 5 shows ratings for selected categories for these ten individuals. 

Categories from the schema were selected for Table 5 if at least three subjects 
from one group had an entry for that category, balanced by at most one subject 
from the ether group. Although the criteria for selecting entries differ between 
Tables 4 and 5, it appears that differences across rated expertise are 
substantially larger than those across experience. An examination of Table 5 
shows that substantial differences between those solutions judged high and low 
can be seen in problem diagnosis, in how a strategy would be developed, in the 
approach to the library holdings and timetable, in the emphasis placed on getting 
to know people, and in the handling of Miss MacDonald. These categories appear, 
at least in part, to capture some of the elements of expertise displayed by the 
subjects in this study. 

************************* 

Insert Table 5 about here 

************************* 
Table 6 offers another perspective on the differences between those 
transcripts scored high and those scored low. Measured in numbers of statements, 
the transcripts of the subjects rated low were only 62% as long as those rated 
high. This is reflected in the number of issues (from the schema) covered and 
the depth of coverage. Table 6 shows the numbers of statements at each rating 
level for the ten individuals in question. Although differences in the groups 
exist in all four rating categories, the largest difference is for level 3, 
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"synthesizing several ideas with evidence of thought". These data as well 
capture an important difference between those rated high and those rated low in 
solution strategy. 

************************* 
Insert Table 6 about here 



Summary and Discussion 
This study arose from ongoing research inuo the analysis of complex problem 
solving in general, and principals' problem solving in particular. Earlier work 
had demonstrated the feasibility of applying schoma theory, in the form of a 
hierarchical category system, to such data. However, this earlier analysis also 
demonstrated that such a system became unwieldy before sufficient differentiation 
between categories. The proposed solution, an application of Biggs and Collis' 
(1982) work, has been successful. The combination of schema analysis and rating 
of statements within each category of the schema has enroled identification of 
factors separating the experienced from the inexperienced, and those judged as 
performing well from those not so judged. As well, although not discussed above, 
detailed examination indicates that factors differentiating subjects on the 
experience dimension differ from those differentiating them on the expertise 
dimension. 

Several issues raised in earlier work and in this study warrant summarizing 
and discussion: 

First, case studies differ from real-life problems; the most noticeable 
differences centre on recognition of and strategies for collection of 
missing information; 
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.Second, the analysis developed is at least partially context dependent; 
understanding of the extent of this dependence would be enhanced by the 
study of true novices (our team is currently analysing data produced by 
pre-service student teachers) and b Y further investigation of other 
problems; 

Third, the reliability of such analytic processes needs to be further examined; 
because of the size of the data set, the development and application of 
the schflma was an iterative process, and reliability may have suffered; 

Fourth, the warm-up, fact-finding, and debriefing aspects of the data collection 
strategies were necessary to insinuate the subjects into the world of Miss 
MacDonald; further attention to the problem of differential adherence to 
think-aloud instruction is required; 

Fifth, this attempt to produce a rating scale for responses produced a variable 
amalgam of a qualitative and a quantitative scale depending on the 
category; it was easier to be eloquent when discussing the role of the 
library than how to build a better library collection; the extent to which 
this is a major issue requires further examination; 

The implications of this study are both theoretical and practical. 
Theoretically, the issue addressed by this study is whether schema theory in 
combination with a rating scale, as adapted, can capture the essentials of 
responses to complex social problems. The results show tnis to be the case. 
Application of such methods to substantial data sets is possible, and results are 
encouraging. 

Practically, the paper raises the question of whether these thods can be 
applied to assessment of outcomes of schooling. Certainly, there appears no 
reason why these methods cannot be applied to written as well as verbal student 
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output. At issue is whether approaches to ill -structured problems based on 
cognitive analysis can make contributions to measurement similar to those made 
in the context of more limited tasks (Snow and Lohman, 1988). If we assume that 
teachers routinely make distinctions of the precision required for the 
categorization reported, then the questions that require pursuit centre in ci.s 
short term on how cognitive science might be applied to develop better marking 
schemes, and in the long term on how such approaches might contribute to the 
development of machine -marking of verbal material. 
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Appendix A 
Miss HacDonald A Case .dy 



Please assume the role of Pat Jones in the following case. Please read out 
loud and verbalize your thoughts as you go, then continue think ing out ^ until 
you have gone as far as you feel you reasonably can in handling this problem. 

Suear TaDle P.S. is an eighteen room elementary school (grades JK - 8) 
located !n a "lower -middle class area of a small city. The building is rather 
old it was built as a high school and was rebelled extensively to accommodate 
the 'elementary school. Pat Jones was appointed principal in August and curing 
the first few days in the school, before opening in September, Pat visited the 
librarv which is located upstairs at one end of the building. Pat noticed that 
not only were the library quarters quite small but there appeared to be few 
modern or new books likely to meet the interests of the older students During 
the first w*ek of school Pat suggested to Miss MacDonald, the school teacher- 
librarian, that she drop into the office to Calk about the library. 

Before the meeting Pat Jones had time to check into some of Kiss 
MacDonald's background and found that she was starting her fifteenth year as 
school librarian; there were instances of personality clashes with other 
teachers- the grade 7 teacher said that her students did not use the library as 
there were ^suitable books; the part-time secretary assigned to the library 
no Wer worked there since a "blow-up" had occurred over whether or not she 
snouU erase pencil marks from the books; and toward the end of the previous 
year Hiss MacDonald had requested to be transferred from .:e library to a 
classroom in another school. 

When Miss MacDonald came in for their meeting, Pat Jones told her that she 
had not received her transfer because she had submitted her request too late, but 
that if she made her request earlier this year there might be a good ch^ce that 
it would be granted. Pat's philosophy is that professionals should be free to 
run their o4 show", and that Hiss HacDonald should feel that she had a free 
h Tnd Pat expressed the hope, however, that the library would operate for the 
benefit of sta^f and students with students encouraged to use the library for 
individual research projects. Hiss HacDonald was highly nervous and restless; 
her sole comment was that her main problem was the large number of books stolen 
each year Pat Jones suggested that possibly she would like to use part of the 
next staff meeting to discuss with the staff ways in which the library could be 
used Miss HacDonald remarked that she did not like talking at staff meetings 
and "as she backed out of the office, she mentioned that she wo J send out a 
form on which teachers could indicate their preference for their scheduled 
library periods. 

Two weeks later. Hiss Green, a first year teacher with a grade seven class, 
complained to Pat Jones that Hiss HacDonald had told her not to leave her 
children unattended in the library. Hiss Green felt that th:s was unfair because 
Hiss HacDonald was telling other teachers when they came to library period, to 
take a prep period and have coffee." She felt she needed e/.tra preparation time 
as much as anybody. Pat Jones told Hiss Green that library periods were not 
"tended for teacher preparation periods; the school had a Partners in Action 
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program in place, and classroom teachers were supposed to be working toother 
with the librarian to plan the use of the library resources as part of the total 
program Kiss Green looked a little sheepish but left the offlc. 
dLfatisfied. Pat Jones was concerned that the library problem was bigger than 
it originally looked, and determined to do something about it immediately. 

At the next staff meeting, Pat Jones introduced the topic of the library. 
Several teachers! apparently representing the majority, declared that there was 
noting wrong with tne library or the way it had always 

that although they valued the scheduled library periods, they would appreciate 
morr freedom to send individuals and small groups to the library for special 
orolects and assignments. Miss MacDonald expressed concern about who would 
supervise these students and went on, in an obviously emotional state, to express 
S f that there should be more co-operative projects ^/^^^ 
"ihrarian- she felt that staff members were not making use of her ser ~ces, 
^omeS^s'she felt she was not earning her salary. Someone asked her vhat she 
mean' c -operative projects; someone else asked for examples of services she 
coSd offer to staff. Miss MacDonald, who wcs extremely nervous by now, was 
unaMe to answer one of the older staff men* ■ ^ suggested, in friendly way, 
that Miss MacDonald should consider taking , .dership training course. 

The meeting ended very soon thereafter, wi.n no agreements or decisions 



made . 



You are Pat Jones. How will you handle the situation? Please think out 
loud as you consider thi* situation your reactions to it. 
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Appendix B 

A Narrative Version of the Schema 1 

Most subjects spent some time talking about the problem at the most global 
level (1) They chose to speak about the "true nature" of the problem (1.1), and 
often about their own problem solving processes and preferences (1.2). Some felt 
that it was a symptom of underlying issues such as morale (1.1.1), others that 
it „as several problems (1.1.4). and still others that it was a straightforward 
and localized issue (1.1.5). Some felt that more data were needed to determine 
if there really was a question to be addressed (1.1.2), while others felt either 
that something needed to be done, or that the wrong action could make things 
worse (11 3). Those who spoke of the problem solving process talked generally 
of how they planned for action (1.2.1), of the role past experiences play in 
their problem solving (1.2.2), of the importance of information and how to get 
it (12 3), or of the development of action plans and/or subproblem 
identification (1.2.4). Some spoke of developing staff ownership of problems as 
a basic strategy (1.2.5). 

A few respondents related the problem to the wider community (2) : id school 
system (3). Some expressed a lack of information about the community (2.3), 
while others thought that community or student input to the library operation 
would be desirable (2.1). Some traced the library problems to a deterioration in 
community morale (2.2). In the context of the school district, a couple of 
sublects asked about board policy (3.1), and a similar number talked of 
consulting with their colleagues (3.2). Most of the references to the school 
system, however, were calls for support staff assistance (3.3 and 3.4). 

About half the respondents made some reference to tne wider school context 
of the problem (4). These responses were of four kinds, concerning school goals 
(4.1), school atmosphere (4.2), the timetable (4.3), and the role of the vice- 
principal (4.4). 

All of the subjects dealt with the remaining three categories of the 
schema the library (5), personnel (6), and the librarian herself , Miss MacDonald 
(7) The statements concerning the library fell into three categories, defining 
the'role of the library (5.1), discussing present practice (5.2), and improving 
the library operation (5.3). Those dealing with the role of the library talked 
of their own expectations (5.1.1), provincial policy (5.1.2), and finding out 
what the staff expected of a library (5.1.3). Those who discussed present 
practice either diagnosed on the b*sis of information given (5.2.1) or outlined 
further information they might need and how they would get it (5.2.2V The 
discussions of library improvement focused on policy implementation (5.3.1, the 
largest category), the library collection (5.3.2), organization and procedures 
(5.3.3), the location and appearance (5.3.4), or the image of the library 
(5.3.5)! 

Statements concerning the staff were categorized into four larger areas: 
gaining entry as a new principal (6.1), building trust and avoiding conflict 



1 The bracketed numbers refer to the elements in Table 1 
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(6 2 ) running a successful staff meeting (6.3). and. the largest, staff 
development (6 4). The entry issues were either diagnosing the difficulties the 
principal L the case caused him or herself (6.1.1) or outlining how to get to 
know people better (6.1.2). both personally and professionally The discussions 
nf trust and conflict were either comments on the importance of the issue (o.Z.l) 
o %Z\fic belies for practice (6.2.2) Discussion 

were either explanations of what went wrong in the case (6.3.1) or strategies for 
doing a better job (6.3.2). The largest personnel category, staff development 
fell into five sub - categories : tea* building (6.4.1) and leadership development 
(6 4 2) causing change (6.4.3). dealing with Miss Green the case study 
character who had the blow-up with the librarian (6.4.4). and the problem of 
older, entrenched teachers (6.4.5). 

The last section of the schema dealt with the librarian herself. Miss 
MacDonald (7) and was divided into four major sections, diagnosis of the problem 
n 1) further data collection (7.2). dealing with her request for a transfer 

7 3) and supporting her towards improvement (7.4). The diagnoses were positive 
or "neutral (7.1 D . negative (7.1.2). or concerned with the possible causes of 
£er revest for a transfer (7.1.3). Further data collection involved asking her 
about her personal feelings (7.2.1) or her plans for the library (7.2.2). Some 
decided they would ask others about her (7.2.3). and others felt they wanted to 
observe her in action or examine her files (7.2.4) . The responses to her request 

„ transfer were to g^t her out for her own good (7.3.1). to remove her for the 
la Zf the sc hool (7 3.2). or to remove her only after possible remediation had 
TiLd (7 3 .3) The discussion of support fell into four categories: personal 
su PP ort and confidence building (7.4.1). help with interpersonal skills (7.4.2). 
improvement of library skills (7.4.3). and development of her leaaership 
potential (7.4.4). 
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Table 1 

Elements gf * Schema 



1 PROBLEM SOLVING 
1 1 Definition of the problem 

1.1.4 several problems thin- library problem 

1.1.5 library or a personnel -within u.Dt«jr v 

1.2 Problem solving process 

i.2.1 Planning visualization 
12 11 need for perspective, time, visua 
1.2 1.2 brainstorming, evolution of plans 

12? Role of Experience 
12 2 1 reflections on role of experience 
1.2 .2.2 specific incident from past 

1.2.3 Role of Data 

1.2.3.1 data collection strategies 

1.2.3.2 data required 
19 4 Problem Elements 

12 4 1 set some priorities, action plan 
1.2.4.2 identify or tackle bits 
1.2.5 Staff Involvement 

1.2. 5.1 develop ownership 

1.2.5.2 delegate, share 

2 COMMUNITY 

2.1 Seek community and student input to library 

2 2 Build community morale 

2.3 Investigate nature of community 

3 SYSTEM 

3.1 Ask about board policy or procedure 

3 2 Consult with colleague 
•»'.3 Bring in library resource 

3 4 Ask about personnel resource 
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Table 1 (cont'd) 



4 SCHOOL 

4.1 School goals 

4.1.1 Jointly establish school goals 
4»1.2 Set expectations 

4.2 Atmosphere of school 

4.2.1 Improve 

4.2.2 Examine 

4.3 Timetabling 

4.3.1 Improve 

4.3.2 Examine 

4.4 Vice- Principal 

4.4.1 Set expectations with/for 

4.4.2 Get info from or use 
Table 1 (cont'd) 

5 THE LIBRARY 

5.1 Role of the library 

5.1.1 State specific expectations (incl. PIA) 

5.1.1.1 State disagreement with own show 

5.1.1.2 State agreement with own show 

5.1.2 State Partners in Action policy 

5.1.3 Seek input of staff expectations 

5.2 Present practice 

5.2.1 Diagnosis of present practice 

5.2.1.1 Problem, Miss M 

5.2.1.2 Problem, older staff 

5.2.1.3 Problem, Miss G 

5.2.1.4 Problem, school or administration 

5.2.2 Data gathering, present practice 

5.3 Improvement 

5.3.1 Policy implementation 

5.3.1.1 Initiate demonstrations and examples 

5.3.1.2 Scheduling and compromise 

5.3.1.2.1 Imposed 

5.3.1.2.2 Negotiated 

5.3.1.3 Staff professional dp^ciopment 

5.3.1.3.1 Principal owned 

5.3.1.3.2 Staff owned 

5.3.1.4 Miss M plans and ideas 
5. 3.1. A. 1 Required to staff 
5.3.1.4.2 Requested to principal 

5.3.1.5 Active principal leadership 
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Table 1 (cont'd) 

5.3.2 Holdings 

5.3.2.1 Discuss inventory, culling, budget history 

5.3.2.2 Increase holdings 

5.3.2.3 Tie to needs or interests 

5.3.3 Organization 

5.3.3.1 Theft 

5.3.3.1.1 Discuss 

5.3.3.1.2 Improve procedures 

5.3.3.2 Get secretarial/volunteer assistance 

5.3.4 Location and appearance, school plant 

5.3.4.1 Noting 

5.3.4.2 Changing 

5.3.5 Image improvement 

6 PERSONNEL 

6.1 Entry, data gathering and rapport 

6.1.1 Diagnosis 

6.1.1.1 lateness 

6.1.1.2 unfamiliarity 

6.1.2 Strategies 

w. 1.2.1 Familiarizing (personal) 
6.1.2.2 Interviewing (professional) 

6.2 Conflict, trust 

6.2.1 Importance 

6.2.2 Strategies 

6.3 Staff meetings 

6.3.1 Diagnosis 

6.3.1.1 Miss K on the spot 

6.3.1.2 Bad meeting 

6.3.2 Strategies 

6.3.2.1 Plan, have data 

6.3.2.2 Other 

6.4 Staff development and supervision 

6.4.1 Team building 

6.4.2 Leadership development 

6.4.3 Change 

6.4.3.1 Time comments 

6.4.3.2 Encourage 

6.4.3.3 Build from example 

6.4.3.4 Build from key staff 

6.4.4 Miss G 

6.4.4.1 Neutral and positive diagnoses 

6.4.4.2 Negative diagnoses 

6.4.4.3 Data required 

6.4.5 Entrenchment (incl. Miss M when appropriate) 
6.4 5.1 Diagnoses 

6.4.5.2 Strategies 

6.4.5.2.1 Develop 

6.4.5.2.2 Move 



ERLC 



31 



Table 1 (cont'd) 



7 MISS MCDONALD 

7.1 Diagnosis 

7.1.1 Neutral or positive 

7.1.2 Negative 

7.1.3 Transfer request 

7.2 Data collection 

7.2.1 Feelings & perceptions from Miss M 

7.2.2 Library plans from Miss M 

7.2.3 Skills & problems from others 

7.2.4 Observations, files 

7.3 Transfer resolution 

7.3.1 Soon, supportive reasons 

7.3.2 Soon, program reasons 

7.3.3 Only after remediation 

7 . 4 Support 

7.4.1 Happiness and confidence 

7.4.1.1 From self 

7.4.1.2 From others 

7.4.2 Group and interpersonal skills 

7.4.3 Library skills 

7.4.3.1 Non-specific 

7.4.3.2 In-school specific 

7.4.3.3 Visits & conferences 

7.4.3.4 Use of authority 

7.4.4 Leadership development, ownership 



Table 2 

Sa agBag of S£a£Slsn£ Ratine s by Bxperte nce LfigSl (Numbers af Sta .r JJ8fin £al 

Rating 1 2 3 4 

Experience 
Level 

Aspirants 38 126 38 6 

Novices 36 110 « U 

Seasoned 1 36 103 61 9 

46 101 34 11 



Veterans 



Total I 56 



440 176 37 



'Note: there were only seven respondents in this group; all other groups had 
eight. 
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Table 3 

Em phasis or ftlohal Sc h ema Categories bv_E3Perjepc e Level 
(per centages o f T nf T fl1 Rating Point3) 

Experience 

Level 12 3 

Schema Category 



Problem Solving 14 
Community 



System 1 
School 2 



Library 



28 



Personnel 21 
Miss MacDonald 31 



14 17 18 



2 112 
1 3 1 

3 3 3 

32 22 16 



20 25 28 

29 28 22 
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Table 4 

Selected Sch™* Elements b y Bvpprlence Level (Total number of Rating Points) 



1. Problem Solving 

1.2.1.1 need for perspective, time, visualization 

1.2.1.2 brainstorming, evolution of plans 

1.2.2.1 reflections on role of experience 

1.2.2.2 specific incident from past 

1.2.3.1 data collection strategies 

1.2.3.2 data required 

1.2.4.1 set some priorities, action plan 

1.2.5.1 develop ownership 

4. School 
4.4.2 Get info from or use v-p 

5. The Library 
5.1.2 State Partners in Action 1 policy 

5.2.1 Diagnosis of present practice 

5.2.1.1 Problem with Miss M 

5.2.1.2 Problem with older staff 
5 3.1.3 Staff professional development 

5.3.1.3.1 Principal owned 

5.3.1.3.2 Staff owned 

6 Personnel 

6.1.2 Entry Strategies 

4.1.2.1 Familiarizing (personal) 

6.1.2.2 Interviewing (professional; 
6.3.2 Staff meeting strategies 

6.3.2.1 Plan, have data 
6.4. 1 Team building 

7. Miss MacDonald 
7.2 Data Collection 

7.2.1 Feelings & perceptions from Miss M 
7.4 Support 

7.4.3 Library Skills 

7.4.3.2 In-school specific 

'Partners in Action is the Ontario school library policy document. 



Experience 


Level 




1 


2 


3 


U 


11 


h 


2 


5 


0 


0 


7 


8 




Z 


16 


2 


4 


7 


11 


16 


3 


ft 


8 


3 


11 


6 


3 


3 


7 


7 


3 


1 


2 




2 


2 


2 


3 


0 


0 


13 


U 


8 


5 


Z 


I z 


0 


0 


8 




0 


2 


11 


7 


1 


0 


2 


7 


0 


0 


8 


2 


10 


12 


5 


9 


13 


8 


2 


3 


1 1 


9 


1 


2 


3 


10 


9 


7 


17 


9 


8 


9 


4 


i 



Table 5 

Selected Schen- ^°"°r^ £<S Tnriivlduals Scored tUsh and Low 



1.1.1.2 Symptom of morale 

1.1.4 several problems 

1.1.5 library c-\ly problem 

1.2.4.1 priorities, plans 

1.2.4.2 identify or tackle bits 

5.3.1.2.2 Negotiated timetable 

5.3.2.1 Discuss library holdings 

5.3.2.2 Increase library holdings 

5 3.2.3 Tie library holdings to needs 



6.1.2.1 
6.1.2.2 
6.4.1 

7.2.1 

7.2.4 

7.3.3 

7.4.1.1 

7.4.2 

7.4.3.3 

7.4.4 



Personal familiarization 
Professional familiarization 
Team building 

Feelings from Miss M 
Observe, get data from iiiles 
Hove only after remediation 
Personally support Miss M 
Develop Miss M's social skills 
Visits & conferences 
Leadership, ownership 



Low 



1 2 3 4 5 



4 
1 



2 3 



2 
2 
3 



High 
1 2 3 4 5 



3 
4 



2 
4 



i 

3 

1 

4 



2 3 

2 
1 
3 



2 3 
3 3 2 



2 
2 
2 
4 



2 
2 



2 
3 



3 
2 
3 
3 
3 
3 
2 



1 
1 

3 
3 

3 
2 



3 
2 



Table 6 

Hu^bers of Statement ^ £ F«rh f Tin, T-evel by fljgb and Log Subjects 



Rating Level 
Low Subjects 

0 

2 2^21 

3 5 9 2 n 
7 1 13 4 0 

* ; fl 7 2 

0.6 

Mean 



1 


9 


3 


7 


11 


2 


2 


8 


2 


5 


9 


2 


1 


13 


4 


9 


8 


7 


4.8 


9.8 


3 



High Subjects 

1 3 11 8 7 

2 6 21 10 1 
^ 4 10 10 0 



5 

Mean 



3 18 13 2 

4 6 14 7 2 



4.4 



14.8 9.6 2.4 
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